AFRL-MN-EG-TN-2007-7009 


Adaptive  Control  Responses  to  Behavioral  Perturbation 
Based  Upon  the  Insect 


R.E.  Ritzmann 
R.D.  Quinn 
M.A.  Willis 

Case  Western  Reserve  University 
10900  Euclid  Ave. 

Cleveland,  OH  44106-4901 


Chris  E  Perry 

Munitions  Directorate 

Air  Force  Research  Laboratory 

Eglin  AFB  FL  32542 

CONTRACT/GRANT  NO.  F08630-03- 1-0003 
NOVEMBER  2006 

Final  Report  for  period  March  2003  -  September  2006. 


DISTRIBUTION  A:  Public  releasable;  distribution  unlimited.  AAC/PA  Approval 
and  Clearance  #  01-04-07-002. 


AIR  FORCE  RESEARCH  LABORATORY,  MUNITIONS  DIRECTORATE 


■  Air  Force  Materiel  Command  ■  United  States  Air  Force 


■  Eglin  Air  Force  Base 


NOTICE  AND  SIGNATURE  PAGE 


Using  Government  drawings,  specifications,  or  other  data  included  in  this  document  for  any  purpose 
other  than  Government  procurement  does  not  in  any  way  obligate  the  U.S.  Government.  The  fact  that 
the  Government  formulated  or  supplied  the  drawings,  specifications,  or  other  data  does  not  license  the 
holder  or  any  other  person  or  corporation;  or  convey  any  rights  or  permission  to  manufacture,  use,  or 
sell  any  patented  invention  that  may  relate  to  them. 

This  report  was  cleared  for  public  release  by  the  Air  Armament  Center  Public  Affairs  Office  and  is 
available  to  the  general  public,  including  foreign  nationals.  Copies  may  be  obtained  from  the  Defense 
Technical  Information  Center  (DTIC)  (http://www.dtic.mil). 


AFRL-MN-EG-TN-2007-7009  HAS  BEEN  REVIEWED  AND  IS  APPROVED  FOR  PUBLICATION  IN 
ACCORDANCE  WITH  ASSIGNED  DISTRIBUTION  STATEMENT. 


FOR  THE  DIRECTOR: 


/  /  original  signed  /  / 


/  /  original  signed  /  / 


FREDERICK  A.  DAVIS,  DR  IV 

TECHNICAL  DIRECTOR 

ASSESSMENT  AND  DEMONSTRATION  DIV 


CHRIS  E  PERRY,  DR  III 
PROGRAM  MANAGER 
MICRO  MUNITION  TECHNOLOGY  TEAM 


This  report  is  published  in  the  interest  of  scientific  and  technical  information  exchange,  and  its 
publication  does  not  constitute  the  Government’s  approval  or  disapproval  of  its  ideas  or  findings. 


n 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and  maintaining  the 
data  needed,  and  completing  and  reviewing  this  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for  reducing  this 
burden  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302. 
Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid 

OMB  control  number.  PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 

1 .  REPORT  DATE  (DD-MM-YYYY)  2.  REPORT  TYPE 

03-11-2006  Final 

3.  DATES  COVERED  (From  -  To) 

March  2003  -  Sept  2006 

4.  TITLE  AND  SUBTITLE 

Adaptive  Control  Responses  to  Behavioral  Perturbation  Based 
Upon  the  Insect 

5a.  CONTRACT  NUMBER 

F08630-03-1-003 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

62602F 

6.  AUTHOR(S) 

R.E.  Ritzmann,  R.D.  Quinn,  Chris  E  Perry 

M. A.  Willis 

5d.  PROJECT  NUMBER 

2502 

5e.  TASK  NUMBER 

99 

5f.  WORK  UNIT  NUMBER 

11 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Case  Western  Reserve  Univ  Munitions  Directorate 

10900  Euclid  Ave  Air  Force  Research  Lab 

Cleveland,  OH  44106-4901  AFRL/MNAV 

Eglin  AFB,  FL  32542 

8.  PERFORMING  ORGANIZATION  REPORT 
NUMBER 

RES423238-Adaptive  Control 
Responses 

9.  SPONSORING  /  MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Munitions  Directorate 

Air  Force  Research  Laboratory 

AFRL/MNAV  101  West  Eglin  Blvd; 

Eglin  AFB  FL  32542 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

AFRL-MN-EG 

11.  SPONSOR/MONITOR’S  REPORT 

NUMBER(S) 

AFRL-MN-EG-TN-2  0 07 -7  0  09 

12.  DISTRIBUTION  /  AVAILABILITY  STATEMENT 


DISTRIBUTION  A:  Public  releaseable;  distribution  unlimited.  AAC/PA  Approval  and 
Clearance  #  01-04-07-002 


13.  SUPPLEMENTARY  NOTES 


14.  ABSTRACT 

The  Munitions  Directorate  at  Eglin  AFB  FL  supported  research  at  Case  Western  Reserve  University  (CWRU) 
in  Cleveland  OH  describing  transitional  behaviors  in  insect  walking  and  in  flight.  Quantitative 
descriptions  of  the  complex  decisions  that  a  cockroach  makes  in  deciding  to  climb  over  or  tunnel  under  a 
barrier  allowed  the  examinations  of  discrete  brain  lesions  in  order  to  pin  point  where  and  how  these 
decisions  are  made.  Documentation  of  turning  movements  developed  hypotheses  regarding  how  descending 
cues  might  alter  local  reflexes  to  turn  the  animal  while  retaining  stability.  Lesions  in  the  brain  and 
subsequent  recordings  demonstrated  how  the  descending  commands  might  be  formulated  within  the  brain.  A 
robotic  leg  was  developed  (hardware  model  of  the  system)  and  could  play  a  critical  role  in  testing  our 
hypotheses  at  a  systems  level .  The  obtained  results  also  positioned  CWRU  to  move  ahead  with  new  funding 
initiatives.  The  brain  recording  and  stimulation  projects  that  were  initiated  during  this  research 
effort  are  now  funded  by  an  NSF  grant  to  the  Ritzmann  laboratory  at  CWRU.  In  addition ,  the  projects  that 
were  undertaken  to  understand  the  alterations  that  occur  at  the  local  control  level  in  response  to 
descending  commands  is  being  pursued  as  a  proposal  to  AFOSR . 

15.  SUBJECT  TERMS 

Descending  neuron-control,  orientation,  kinematics,  biorobotics,  cockroach 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION 

OF  ABSTRACT 

18. 

NUMBER 

OF  PAGES 

19a.  NAME  OF  RESPONSIBLE 
PERSON 

Mr.  Chris  E  Perry 

a.  REPORT 

b.  ABSTRACT 

c.  THIS  PAGE 

19b.  TELEPHONE  NUMBER  (include 

UL 

29 

area  code) 

UNCLASSIFIED 

UNCLASSIFIED 

UNCLASSIFIED 

iii 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std.  Z39.18 


Table  of  Contents 


Table  of  Contents . iv 

Executive  Summary . v 

Introduction . 1 

Scientific  Approach . 1 

Background  Information:  Insect  Anatomy . 4 

Arrangement  of  Insect  Thoracic  Nervous  Systems . 4 

Block  Climbing:  An  Example  of  Transitional  Behavior . 6 

Results . 6 

Transitional  Behaviors  in  Cockroach . 6 

Local  Control  in  Cockroach  Turning  and  Climbing . 7 

Processing  in  the  Brain . 9 

Motor  Activity  Associated  with  Flight  Turns . 10 

Implementation  of  Biologically  Inspired  Control  into  Robotic  Systems . 12 

Summary . 15 

References . 16 

Appendix  A:  Research  Output  Generated  During  Period  of  Grant . 18 


IV 


EXECUTIVE  SUMMARY 


One  of  the  distinguishing  features  of  animal  behavior  is  the  capacity  to  respond  quickly  and  efficiently 
to  a  range  of  perturbations  affecting  the  animal’s  ambulatory  progress.  Rather  than  switching  to  new 
control  algorithms,  animals  adapt  to  these  changing  conditions  without  hesitation  by  modifying 
activity  in  existing  neural  circuitry.  If  this  capacity  to  alter  control  in  response  to  ever  changing 
conditions  could  be  captured  in  the  control  architecture  of  artificial  devices  ranging  from  robots  to 
missiles,  it  would  greatly  increase  their  ability  to  perform  missions  independently  under  real  world 
conditions.  However,  we  are  only  beginning  to  understand  how  animals  solve  these  problems.  Insects, 
which  are  the  most  successful  animals  in  the  world  and  certainly  among  the  most  agile,  provide  several 
important  technical  advantages  for  studying  neural  control  of  adaptive  behavior.  It  is  therefore 
believed  that  a  comprehensive  study  of  insect  neural  control  schemes  can  greatly  benefit  designs  for 
man-made  vehicles. 

During  this  technical  effort,  scientists  from  Case  Western  Reserve  University  (CWRU)  investigated 
how  insects  identify  and  characterize  obstacles  and  then  use  that  information  to  command  alterations 
in  forward  locomotion  all  in  a  matter  of  milliseconds.  A  range  of  insect  preparations  already  provide 
considerable  neurobiological  information  on  the  processing  of  sensory  information  within  brain 
regions  and  the  corrective  actions  that  occur  in  local  motor  circuits  found  in  the  thoracic  ganglia. 
However,  very  little  is  known  about  the  linkage  between  these  two  regions  and  it  is  here  that  they 
focused  there  attention.  Although  they  chose  to  focus  upon  walking  insects,  because  of  technical 
advantages  to  these  preparations,  they  contended  that  the  control  rules  that  were  uncovered  could  be 
generalized  to  any  animal  or  vehicle  that  must  move  through  three  dimensions  and  respond  quickly 
and  efficiently  to  barriers  in  its  path.  This  assumption  was  tested  by  comparing  data  from  cockroaches 
that  move  primarily  by  legged  locomotion  with  similar  data  from  moths  that  move  primarily  by  flight. 
Further,  they  incorporated  the  resulting  control  rules  into  both  software  simulations  and  robotic 
systems. 
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INTRODUCTION 


Novel  developments  of  technical  devices  to  solve  our  most  serious  problems  can  benefit  from  an 
intelligent  interplay  between  biology  and  engineering.  Some  of  the  most  challenging  problems  to 
technical  development  have  already  been  solved  by  animals  through  millions  of  years  of  evolution. 
Very  few  vehicles  can  match  the  agility  of  legged  animals  as  they  move  through  complex  natural 
terrains.  Nevertheless,  merely  copying  animal  systems  will  not  provide  optimal  solutions.  After  all, 
manmade  devices  often  surpass  specific  capabilities  of  any  animal  system.  For  example,  man  has 
managed  to  develop  devices  that  are  more  powerful,  faster  and  can  fly  higher  than  any  animal  could 
every  hope  to  achieve.  Moreover,  issues  of  scaling  and  materials,  make  direct  mimicry  a  less  than 
optimal  strategy.  Rather,  what  is  required  is  an  intelligent  strategy  for  lifting  rules  from  biological 
systems  and  incorporating  them  appropriately  into  control  architecture  for  robotic  devices. 

The  capabilities  of  animals  have  lead  several  groups  to  examine  biological  systems  as  inspiration  for 
agile  and  efficient  man-made  vehicles.  The  strategies  that  have  been  employed  in  these  studies  vary 
greatly  (Ritzmann,  Quinn,  Watson  and  Zill,  2000).  On  one  hand,  biologists  may  make  detailed 
observation  of  relevant  aspects  of  movement  and  provide  these  data  to  engineers  who  implement  them 
as  much  as  is  possible  in  their  devices  in  order  to  capture  the  animal’s  dexterity.  At  the  other  extreme, 
the  engineers  design  a  device  using  standard  engineering  practices  and  then  turn  to  biological  systems 
as  a  resource  for  solutions  to  specific  problems.  In  either  of  these  cases,  the  specific  goal  is  the 
development  of  improved  devices. 

The  reverse  relationship  has  also  been  fruitful.  Robotic  devices  have  been  developed  by  several 
biological  groups  specifically  to  test  neural  control  and  mechanical  hypotheses.  Webb  (2001)  utilized 
wheeled  robots  to  test  notions  about  cricket  song  recognition  as  did  Grasso  in  testing  hypotheses 
regarding  olfactory  guidance  in  lobsters  (Grasso,  Dale,  Consi,  Mountain  and  Atema,  1996).  Srinivasen 
implemented  his  notions  about  visual  navigation  in  bees  on  robotic  devices  to  test  their  real  world 
capabilities  (Srinivasan  and  Zhang,  2000;  Srinivasan,  Zhang  and  Chahl,  2001).  Dickinson  built  a 
robotic  device  to  measure  forces  associated  with  flight  behavior  in  insects  (Dickinson,  Lehmann  and 
Sane,  1999). 

CWRU  proposed  to  span  both  of  these  goals,  by  building  upon  the  unique  biology-engineering 
relationship  that  has  developed  over  15  years  at  Case  Western  Reserve  University.  The  insect 
locomotion  studies  that  were  proposed  initially  emphasized  biological  questions.  However,  the 
resulting  control  rules  were  implemented  in  existing  robotic  devices  via  dynamic  simulation  tools  that 
are  currently  available  to  our  group.  This  effort  will  not  only  result  in  more  capable  devises,  but  also 
provided  hardware  models  for  generation  of  new  hypotheses  that  can  be  tested  by  further  biological 
experimentation.  In  this  way,  the  biological  data  will  lead  to  engineering  advances  that  will  lead  back 
to  greater  understanding  of  the  original  biological  systems. 


SCIENTIFIC  APPROACH 

Natural  environments  are  heterogeneous  and  dangerous  places.  Although  some  natural  conditions  can 
be  tamed  by  removing  trees  and  building  flat  roads,  we  cannot  remove  all  barriers.  Consequently, 
locomotion  through  natural  terrain  requires  rapid  decision  making  and  quick  alterations  in  movement 
trajectory.  Most  animals  handle  these  problems  very  efficiently.  In  particular,  insects  can  run  or  fly 
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smoothly  through  environments  that  are  strewn  with  objects  that  often  require  them  to  rapidly  react  to 
barriers  in  order  to  reach  their  ultimate  goals. 

A  considerable  amount  of  effort  in  motor  control  has  rightly  focused  upon  the  basic  steady-state 
movements  that  animals  use  to  move  across  a  flat  surface.  Unfortunately,  relatively  little  attention  has 
been  paid  to  the  transitional  behaviors  that  allow  animals  to  efficiently  negotiate  natural  environments, 
in  spite  of  the  fact  that  it  is  an  understanding  of  these  very  responses  to  perturbation  that  would  have 
the  greatest  impact  upon  the  development  of  man-made  devices.  Rapid  adjustments  approaching 
animal-like  responses  would  clearly  benefit  the  design  of  autonomous  vehicles.  Currently,  most  of 
these  problems  are  handled  by  a  driver,  pilot  or  tele-operator.  It  is  not  only  inefficient  to  rely  on  a 
person  to  make  all  these  adjustments,  but  in  many  conditions  the  delays  may  be  so  long  that  a  rapidly 
moving  vehicle  would  crash  or  fail  its  mission  before  the  adjustment  is  made.  Devices  that  are  moving 
through  dangerous  regions  may  move  slower,  but  this  advantage  is  lost  by  even  greater  delays  in 
communication  between  the  device  and  a  driver  positioned  at  a  safe  distance  from  the  operation. 
Consider  a  robotic  vehicle  searching  for  survivors  in  a  collapsed  building.  If  the  robot  encounters  a 
barrier  to  its  movement,  but  must  rely  on  a  signal  to  a  tele-operator  outside  the  building,  it  may  crash 
or  become  irreversibly  stuck  before  a  sensor  reports  the  problem  to  the  operator,  he  or  she  makes  a 
decision  and  the  correction  signal  reaches  the  robot.  Similar  issues  are  faced  by  both  missiles  and  their 
targets  as  they  enter  the  endgame  strategies  of  pursuit  (Cloutier,  Evers  and  Feeley,  1989).  By 
empowering  control  systems  with  animal-like  responses,  reaction  time  would  be  greatly  reduced  and 
problems  posed  by  barriers  and  other  perturbations  could  be  easily  avoided. 

In  the  past,  CWRU  has  focused  upon  transitional  behaviors  in  walking  insects,  because  a  considerable 
amount  of  information  is  already  available  for  these  systems  and  the  necessary  biological  experiments 
can  be  readily  performed  on  cockroaches.  These  advantages  make  walking  insects  particularly 
attractive  for  the  kinds  of  studies  that  we  are  proposing.  Nevertheless,  it  should  be  possible  to 
generalize  the  control  rules  that  result  from  these  studies  to  any  animal  or  vehicle  that  moves  through 
three  dimensions  and  encounters  obstacles  in  its  path.  To  do  this,  one  must  identify  the  components  of 
the  animal  or  vehicle  that  generate  thrust,  turning  (yaw)  and  pitch  as  well  as  appropriate  sensors  for 
detecting  and  characterizing  the  barriers.  One  must  then  characterize  the  link  between  detection  and 
actuation. 

To  test  this  notion,  CWRU  compared  the  results  from  their  walking  studies  to  control  in  flying  insects. 
This  was  done  in  two  ways.  First,  control  of  wing  movements  in  Blaberus  discoidalis,  the  same  insect 
that  is  being  studying  for  walking,  was  examined.  Blaberus  will  readily  generate  wing  movements  in 
response  to  wind  directed  at  its  head.  Therefore,  the  role  of  individual  intemeurons  during  walking 
and  flying  within  the  same  animal  could  be  examined.  However,  Blaberus  does  not  fly  freely  very 
well.  Therefore,  the  results  on  data  taken  on  the  hawkmoth,  Manduca  sexta,  which  is  a  very  strong 
flyer  and  is  comparable  in  size  to  Blaberus  discoidalis ,  was  also  compared.  A  thorough  examination 
of  descending  control  for  both  walking  and  flying  behaviors  was  beyond  the  scope  of  a  three  year 
project  and  CWRU  was  much  further  along  in  walking  than  in  flying.  Therefore,  CWRU  concentrated 
their  efforts  on  transitional  behaviors  involved  in  walking,  while  examining  flying  in  sufficient  detail 
to  test  the  notion  of  generalization. 

The  control  regions  for  insect  locomotion  can  be  broken  up  into  three  regions  (Fig.  1).  The  sensors 
located  on  the  head  detect  barriers  to  forward  locomotion  and  project  to  brain  regions  were  they  are 
processed  and  integrated  with  information  from  other  sensors  and  data  on  motor  activities.  Based  upon 
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this  information,  commands  are  formed  in  the  brain  and  sent  via  the  cervical  connectives  to  local 
motor  control  regions  in  the  thoracic  ganglia.  The  head  sensors  (antennae  and  visual  systems)  are 
comparable  in  the  cockroach  and  the  hawkmoth  (Fig.  1).  The  motor  commands  descend  through  a 
limited  number  of  cervical  intemeurons  to  local  motor  control  systems  in  thoracic  ganglia.  Here 
differences  will  occur.  In  the  cockroach  the  local  systems  will  control  six  unique  pairs  of  legs  that  alter 
movements  to  generate  variations  in  turning  (yaw),  body  attitude  (pitch),  and  forward  speed  (thrust). 
In  the  moth,  alterations  in  wing  movement  will  generate  similar  motor  effects  as  the  animal  flies. 
Similarly  wing  control  in  the  cockroach  will  employ  different  sets  of  motor  neurons  and  muscles  than 
those  that  control  the  legs. 


Brain  Cervical  Thoracic 

Connectives  Ganglia 


Figure  1.  Hypothetical  arrangement  of  control  elements  for  transitional  behaviors  involved  in  flight 
and  walking. 

Clearly,  a  total  understanding  of  the  issues  presented  by  either  walking  or  flight  control  is  beyond  the 
scope  of  any  short  term  investigation.  Fortunately,  many  other  investigators  are  currently  working  on 
several  aspects  of  the  problem.  New  information  is  becoming  available  on  sensory  processing  in  insect 
brains  (Gupta,  1987)  and  reflex  responses  in  motor  circuits  (Bassler,  1998;  Burrows,  1996).  What  is 
almost  totally  lacking  is  an  understanding  of  the  linkage  between  these  two  regions.  Specifically,  the 
question  that  was  pursued  was:  How  do  behavioral  decisions  that  are  arrived  at  in  the  brain  alter 
specific  motor  activity? 

CWRU  proposed  a  comprehensive  set  of  biological  experiments  on  both  walking  and  flying  insects 
that  made  tremendous  strides  in  establishing  how  this  linkage  occurs.  The  resulting  principles  were 
then  modeled  in  dynamic  simulation  and  implemented  in  robotic  systems  that  were  currently  available 
to  the  research  team.  These  latter  aspects  of  the  proposed  work,  allowed  the  test  and  refinement  of  the 
hypotheses  in  real  world  systems,  and  assured  that  the  resulting  system-level  control  principles  could 
be  incorporated  into  a  range  of  man-made  devices  that  have  importance  to  Department  of  Defense 
missions. 

The  specific  aims  of  this  project  were  to: 

1 .  Identify  and  characterize  sensors  on  the  cockroach  head  that  are  associated  with  detection  and 
evaluation  of  objects  that  are  barriers  to  forward  locomotion  and  compare  the  actions  of  these 
structures  to  analogous  sensors  on  the  moth.  The  roles  of  antennae,  ocelli  and  compound  eyes  in 
obstacle  detection  and  evaluation  under  various  conditions  such  as  light  level  and  walking  speed  were 
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examined.  Differences  in  their  functions  under  walking  and  flying  behaviors  were  also  examined. 

2.  Identify  descending  intemeurons  that  conduct  information  and  decisions  from  brain  regions 
that  process  information  from  sensors  on  the  head  to  motor  centers  in  thoracic  ganglia  that  control  leg 
movements  and  compare  to  descending  intemeurons  used  in  flight.  The  observation  that  in  many 
insects  relatively  few  descending  intemeurons  exist  (Staudacher,  1998),  suggests  that  the  same  cells 
may  be  used  for  multiple  behaviors  such  as  walking  and  flying. 

3.  Using  a  tethered  preparation,  characterize  the  motor  changes  that  occur  in  order  to  transition 
from  walking  to  turning  and  formulate  hypotheses  regarding  how  descending  activity  brings  these 
changes  about.  The  hypothesis  suggests  that  this  is  where  walking  and  flying  control  diverges.  By 
comparing  the  motor  changes  found  in  walking  to  those  in  flying,  it  could  be  determined  whether 
similar  descending  cues  can  access  both  systems  and  if  so,  whether  control  principles  generalize 
according  to  movement  axes  such  as  yaw,  thrust  and  pitch. 

4.  Implement  control  strategies  in  a  range  of  legged  robotic  systems.  By  implementing  the  rules 
that  are  developed  in  this  study  into  control  systems  for  existing  robotic  devices,  the  hypotheses  under 
real  physical  conditions  could  be  tested,  establish  their  capacity  for  use  in  robotic  vehicles,  and 
develop  new  hypotheses  for  further  biological  tests. 


BACKGROUND  INFORMATION:  INSECT  ANATOMY 

Arrangement  of  Insect  Thoracic  Nervous  Systems 

A  complex  mix  of  sensory  inputs  and  endogenous  pattern  generation  circuits  control  movement  in 
virtually  all  legged  animals.  The  neural  control  systems  are  divided  into  specific  locations  within  the 
animal’s  central  nervous  system.  An  insect  has  six  legs  with  each  pair  located  on  a  separate  thoracic 
segment  (Fig.  2)  (Burrows,  1996).  Flying  insects  possess  one  to  two  pairs  of  wings  located  on  the 
second  and  (where  two  pairs  are  present)  third  thoracic  segments.  Within  each  segment  the  nerve  cord 
expands  to  form  a  ganglion  that  contains  approximately  1000  intemeurons  and  motor  neurons  that 
control  movements  associated  with  that  segment. 
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Figure  2.  Diagram  of  cockroach  nervous  system  showing  Brain,  Suboesophageal  ganglion,  Thoracic 
ganglia  and  abdominal  ganglia. 

Each  pair  of  legs  is  controlled  by  a  separate  thoracic  ganglion.  Location  of  two  lesions  are  also  noted 
for  discussion  in  the  following.  A  considerable  amount  of  behavioral  and  neurobiological  data  have 
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been  amassed  regarding  the  neural  circuits  found  in  the  thoracic  ganglia.  These  so-called  local  control 
circuits  contain  the  motor  neurons  that  directly  control  leg  muscles  as  well  as  local  intemeurons  that 
serve  to  group  sensory  information  from  the  leg  and  coordinate  actions  of  the  various  muscles  within 
different  leg  segments  (Burrows,  1989).  These  elements  are  influenced  by  pattern  generation  neurons 
that  set  the  rhythm  for  oscillatory  behaviors  such  as  those  that  initiate  leg  movements  during  walking 
and  running.  There  are  also  interganglionic  intemeurons  that  coordinate  actions  between  each  leg  to 
generate  such  movements  as  the  tripod  gait  (Cruse,  1990;  Schmitz,  Dean,  Kindermann,  Schumm  and 
Cruse,  2001).  Sensory  receptors  located  on  and  within  the  leg  play  important  roles  in  monitoring  leg 
movement  during  walking  (Zill,  1990). 

A  case  can  and  has  been  made  that  much  of  the  control  of  leg  movements  during  walking,  even  over 
uneven  terrain,  can  be  attributed  to  the  local  control  circuits  in  the  thoracic  ganglia  (Cruse,  1990; 
Schmitz,  et  ah,  2001).  Indeed,  the  rules  by  which  this  control  can  be  accomplished  provide  the  basis 
for  locomotion  controllers  of  several  hexapod  robots  (Beer,  Chiel,  Quinn  and  Ritzmann,  1998). 
However,  observations  dating  back  to  the  work  of  Roeder  (1937)  have  clearly  shown  that  insects  do 
not  walk  in  a  normal  tripod  gait  after  lesioning  the  neck  connectives  (cervical  lesion  in  Fig.  2),  thereby 
isolating  the  thoracic  ganglia  from  neural  centers  in  the  head,  including  the  brain  and  suboesophageal 
ganglion.  We  have  repeated  these  experiments  on  animals  that  were  maintained  alive  by  hand  feeding 
for  up  to  two  months  post  lesion  (Ridgel,  Alexander,  Mu,  Ritzmann  and  Strausfeld,  2002).  We  found 
that  over  the  course  of  10  days,  stepping  did  increase,  and  some  coordination  returned  between  left  and 
right  legs  of  individual  thoracic  segments.  However,  intersegmental  coordination  never  returned  and 
joint  movements  were  much  reduced.  Injection  of  the  neuromodulator  octopamine  increases  the 
number  of  steps  taken,  but  does  not  improve  intersegmental  coordination.  Rarely,  step  cycles 
corresponding  to  the  “classic”  alternating  tripod  gait  are  produced.  Thus,  it  appears  that  the  thoracic 
circuitry  that  organizes  action  into  a  tripod  gait  does  exist  in  the  thoracic  ganglia.  However,  that 
coordination  is  not  readily  expressed  without  connection  to  the  higher  centers. 

Roeder  (1937)  also  reported  that  a  lesion  anterior  to  the  suboesophageal  ganglion  (circumoesophageal 
lesion  in  Fig.  2),  which  sits  in  the  head  between  the  brain  and  the  thoracic  ganglia,  results  in  an  animal 
that  walks  constantly.  We  also  confirmed  and  extended  this  observation.  Blaberid  cockroaches  with 
bilateral  lesions  of  the  circumoesophageal  connectives  walk  interminably  in  a  perfect  tripod  gait. 
These  animals  can  move  around,  over  and  under  barriers.  However,  since  they  lack  input  from  the 
sensory  processing  regions  of  the  brain,  they  do  not  do  so  like  an  intact  animal.  Intact  animals  detect 
barriers  through  actions  of  sensors  such  as  eyes  and  antennae.  Posture  is  then  altered  well  before 
reaching  the  barrier.  As  a  result,  they  seamlessly  move  over,  under  or  around  the  block  without 
bumping  into  the  obstacle.  Animals  with  circumoesophageal  connective  lesions  (CoCL)  do  not  show 
this  coordinated  behavior  when  challenged  with  an  obstruction.  Rather,  they  alter  their  movements 
only  after  running  into  the  barrier.  For  example,  they  will  climb  up  a  block  by  simply  pushing  forward 
until  the  body  bends  upward  and  slides  up  the  front  surface. 

In  addition  to  the  obvious  sensory  deficits  described  above,  CoCL  animals  also  have  some  surprising 
motor  problems.  They  have  difficulty  walking  up  inclines,  probably  because  they  do  not  adjust 
walking  speed  to  new  conditions.  They  also  have  difficulty  flexing  thoracic  segments  causing  them  to 
high-center  badly  when  climbing  over  blocks. 

From  these  observations,  it  can  be  concluded  that  neural  circuits  within  the  suboesophageal  ganglion 
promote  forward  locomotion  and  the  expression  of  appropriately  coordinated  leg  movements. 
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Furthermore,  the  normal  transition  from  forward  movement  to  turning  or  climbing  behaviors  in 
anticipation  of  a  barrier  requires  input  from  even  higher  centers  found  in  the  brain.  Thus,  a  critical 
aspect  in  the  control  of  transitional  behaviors  relies  upon  the  neural  interactions  among  brain, 
suboesophageal  ganglion  and  local  control  circuits  in  the  thoracic  ganglia. 

Block  Climbing:  An  Example  of  Transitional  Behavior 

CWRIJ  recently  described  the  kinematics  and  motor  activity  associated  with  one  transitional  behavior 
(Watson,  Ritzmann  and  Pollack,  2002;  Watson,  Ritzmann,  Zill  and  Pollack,  2002).  As  a  cockroach 
approaches  an  11  mm  block,  it  rears  upward  by  rotating  the  middle  legs  and  pushing  downward 
(pitch).  It  can  then  place  its  front  legs  on  top  of  the  block  and  push  its  body  up  and  over  the  obstacle  by 
extending  the  powerful  rear  legs  (thrust).  It  typically  performs  the  critical  rearing  movement  without 
making  significant  contact  with  the  front  legs  along  the  leading  edge  of  the  object.  Thus,  it  appears  to 
detect  and  measure  the  block  using  sensors  located  on  its  head  and  then  to  direct  the  appropriate 
movements  to  efficiently  get  over  the  barrier. 

Block  climbing,  thus,  requires  a  direct  interaction  between  sensors  located  on  the  head  and  sensory 
processing  regions  within  the  brain.  The  resulting  motor  commands  descend  through  intemeurons  in 
the  cervical  connectives  to  thoracic  ganglia  where  they  must  be  interpreted  by  local  neural  circuits 
specific  to  each  motor  plant  (legs  or  wings).  The  local  circuits  each  rely  upon  proprioceptive  reflexes 
to  match  the  resulting  movements  to  ongoing  external  and  internal  conditions.  The  properties  outlined 
here  make  block  climbing  an  excellent  transitional  behavior  on  which  to  focus  for  studies  on  motor 
control  in  complex  terrain. 


RESULTS 

The  “Adaptive  Control  Responses  to  Behavioral  Perturbation”  project  was  very  successful  in 
discovering  new  information  on  how  insects  control  turning  maneuvers.  Great  headway  was  made  in 
both  biological  and  engineering  aspects  of  the  project.  Moreover,  strong  foundations  were  developed 
for  future  studies.  The  results  will  be  described  three  segments.  First,  the  progress  in  understanding 
local  and  descending  control  in  cockroach  transitional  behaviors  will  be  described.  Then  the  progress 
in  understanding  similar  aspects  of  moth  flight,  which  was  included  in  the  proposal  to  test  the 
universality  of  our  hypotheses,  will  be  described.  Finally,  the  progress  in  robotic  implementation  that 
will  ultimately  provide  a  unique  opportunity  for  both  testing  and  implementation  of  our  notions  of 
control  in  complex  behavior,  will  be  covered. 

Transitional  Behaviors  in  Cockroach 

The  goal  for  this  project  was  to  understand  how  the  insect’s  nervous  system  generates  transitional 
behaviors  to  deal  with  barriers  to  forward  locomotion.  Previous  work  clearly  demonstrated  that 
cockroaches  get  around  objects  in  their  path  by  evaluating  the  barrier  with  sensors  on  their  head  and 
using  the  information  that  they  gain  to  alter  leg  movements  (Watson  et  al.,  2002a;  Watson  et  ah, 
2002b).  The  decisions  made  in  climbing  and  turning  behaviors  using  high-speed  video  to  document 
the  insect’s  decisions  were  further  examined.  These  data  were  reported  in  two  posters  at  the  last  two 
meetings  of  the  Society  for  Neuroscience  (Harley  et  al.,  2005;  Harley  et  al.,  2006)  and  are  currently 
being  written  up  for  publication  in  a  journal  article.  Markov  chain  models  demonstrated  that  the 
cockroach  uses  information  gained  by  tapping  the  object  with  its  antennae  to  guide  climbing 
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movements  over  a  block.  If  the  block  is  replaced  with  a  shelf,  the  cockroach  is  now  faced  with  the 
choice  of  either  climbing  over  or  tunneling  under  the  object.  The  choice  taken  is  determined  by 
whether  the  antennae  touch  the  shelf  from  the  top  or  form  underneath.  Tapping  from  the  top  generates 
climbing,  while  touching  from  underneath  causes  tunneling.  Ambient  light  also  seems  to  be  important. 
In  bright  lights,  the  cockroach  tends  to  tunnel  twice  as  much  as  it  climbs,  while  in  low  light  the 
probability  of  either  behavior  occurring  is  equal. 

More  information  about  what  is  being  gained  from  the  antennae  was  documented  by  ablation 
experiments.  Ablation  of  antennae  alters  the  point  at  which  the  climbing  commences.  The  shorter  the 
antennae  are,  the  closer  the  cockroach  gets  to  a  block  before  starting  to  rear  up.  Interestingly,  if  the 
antennae  are  completely  removed,  the  cockroach  completely  alters  its  climbing  strategy.  Rather  than 
using  a  controlled  rearing  behavior,  it  now  uses  elevator  reflexes.  In  this  latter  strategy,  the  cockroach 
bumps  its  leg  repeatedly  onto  the  front  surface  of  the  barrier.  Each  contact  causes  the  cockroach  to  lift 
its  leg  higher  until  it  is  placed  on  the  top  surface  of  the  block.  This  strategy  is  also  used  by  insects 
with  circumoesophageal  connectives  lesioned  to  disconnect  the  brain  from  the  thoracic  ganglia. 

Local  Control  in  Cockroach  Turning  and  Climbing 

At  the  outset  of  the  project,  work  was  started  on  two  distinct  areas  thought  to  be  involved  in  the  neural 
control  systems  underlying  turning  behavior;  (1)  direct  local  control  systems  within  the  thoracic 
ganglia  and  (2)  descending  commands  formulated  within  specific  brain  regions.  To  investigate  these 
issues  in  cockroaches,  a  tethered  preparation  that  allows  us  to  study  turning  movements  in  detail  was 
developed.  The  insect  is  placed  over  a  lightly  oiled  glass  plate  and  forward  movement  is  constrained 
by  two  pins  placed  in  the  pronotum  (dorsal  cuticular  shield).  In  this  condition,  the  cockroach  walks 
normally  but  its  body  remains  in  one  place.  Under  these  conditions,  very  detailed  kinematic  and 
neurobiological  observations  even  extending  to  intracellular  recording  were  made.  It  was 
demonstrated  that  turning  movements  could  be  generated  in  the  tethered  preparation  by  pushing  on  one 
antenna.  Essentially,  the  animal  goes  from  a  very  symmetrical  left-right  set  of  joint  movements  during 
walking  to  a  decidedly  asymmetrical  set  of  movements  during  turning.  In  the  leg  that  would  be  on  the 
outside  of  the  turn,  motor  and  joint  activity  is  similar  to  walking,  but  directed  more  laterally. 
However,  in  the  inside  leg,  the  distal  femur-tibia  (FTi)  joint  increases  extensor  activity  and  joint 
velocity,  while  the  more  proximal  coxa- trochanter  (CTr)  joint  decreases  depressor  motor  activity  and 
shows  very  reduced  joint  movement  (Fig.  3).  Most  importantly,  these  joint  extension  movements  occur 
during  swing  phase  in  the  inside  leg,  rather  than  during  stance  as  in  all  other  walking  movements.  A 
paper  that  describes  these  changes  in  detail  was  published  in  the  Journal  of  Comparative  Physiology  A 
(Mu  and  Ritzmann,  2005).  That  paper  proposes  an  hypothesis  for  how  the  changes  are  generated, 
suggesting  that  descending  commands  alter  critical  parameters  in  the  local  control  reflexes  within  the 
thoracic  ganglia  then  allow  the  sensori-motor  reflexes  to  push  the  system  to  a  new  stable  (turning) 
state. 

Biological  experiments  were  initiated  to  test  this  hypothesis  and  to  developing  a  robotic  device 
(described  later)  that  will  provide  a  unique  opportunity  to  establish  how  realistic  this  explanation  is 
under  real  physical  conditions.  In  the  biological  experiments,  it  was  asked  whether  movements  that  are 
similar  to  inside  turning  can  be  generated  in  animals  when  commands  descending  from  higher  brain 
centers  are  completely  eliminated.  This  was  completed  by  cutting  the  two  circumoesophageal 
connectives  that  link  the  brain  to  the  suboesophageal  ganglion.  These  circumoesophageal  connective 
lesioned  (CoCL)  animals  walk  continuously.  Using  dual  high-speed  video  systems  and  three 
dimensional  motion  analysis  software,  very  careful  comparisons  among  walking,  turning  and  searching 
movement  both  in  intact  and  lesioned  insects  could  be  made.  On  the  tether,  searching  movements  by 
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the  legs  can  be  evoked  by  simply  dropping  the  substrate  away  from  the  animal.  At  the  leg,  this  action 
resembles  the  swing  phase  extension  seen  in  inside  turning.  The  FTi  and  CTr  joint  movements  and 
associated  motor  activity  are  very  similar  to  inside  turning.  Thus,  the  critical  parameter  for  turning 
appears  to  be  the  swing  phase  extension  implying  that  the  primary  target  of  descending  commands 
would  be  muscles  that  lift  the  leg  to  begin  swing.  Additional  experiments  are  currently  being 
performed. 


Tethered  Walking  Turning  Inside  Turning  Outside 


Stance 


Time  (ms) 

Figure  3.  Motor  activity  and  joint  movement  from  tethered  preparation  showing  walking  (first  column) 
and  bilateral  activity  that  occurs  during  turning  (second  and  third  column).  Note  the  change  that  occurs 
in  the  motor  activity  and  joint  kinematics  in  the  inside  leg.  Inset  shows  the  two  joints  depicted  here. 


In  addition  to  the  three  dimensional  reconstruction  of  leg  movements,  students  in  the  Quinn  and 
Ritzmann  CWRU  laboratory  teamed  up  to  develop  a  computer  simulation  of  middle  leg  movements 
that  allowed  us  to  examine  the  effects  of  each  leg  joint  movement  upon  the  actions  of  the  foot.  This 
model  was  presented  at  the  recent  meeting  of  the  Society  for  Neuroscience  and  was  very  enlightening 
(Mu  et  al.,  2006).  Many  of  the  effects  of  joint  changes  between  walking  and  turning  confirmed  our 
previous  hypotheses.  However,  some  very  subtle  effects  were  also  found  to  be  very  important.  For 
example,  in  walking,  the  proximal  CTr  joint  extension  and  Ds  activity  commences  slightly  before  the 
more  distal  FTi  joint  extension  and  its  SETi  motor  neuron. 


In  contrast,  the  inside  leg  of  the  turning  animal  has  this  timing  reversed,  with  the  FTi  joint  preceding 
the  CTr  joint.  This  subtle  change  is  actually  critical  to  the  foot  actions  associated  with  joint  extension, 
which  occurs  in  stance  during  walking  but  in  swing  in  inside  leg  turning  actions.  If  the  CTr  joint 
extends  first  the  foot  drives  into  the  substrate  and  FTi  extension  can  then  only  contribute  to  downward 
force.  However,  if  the  FTi  joint  extends  first,  the  foot  is  directed  upward  contributing  to  the  swing 
phase  extension  associated  with  turning. 


Finally,  it  was  demonstrated  that  descending  signals  from  the  brain  have  profound  influences  upon 

8 


local  control  reflexes  within  the  thoracic  ganglia.  It  was  demonstrated  that  movement  of  the  femoral 
chordotonal  organ  that  monitors  joint  position  of  the  FTi  joint  evokes  activity  in  both  the  motor 
neurons  that  control  that  joint  and  those  that  control  the  next  proximal  CTr  joint.  In  the  latter,  Ds 
motor  activity  is  enhanced  when  the  chordotonal  organ  is  extended  and  inhibited  when  it  is  relaxed. 
These  inter-joint  reflexes  were  previously  described  for  stick  insect  (Hess  and  Biischges,  1999),  but 
this  is  the  first  description  in  cockroach.  In  insects  that  had  experienced  bilateral  lesion  of  the  neck 
connectives  or  the  circumoesophageal  connectives,  thereby  removing  descending  activity,  both  the 
excitatory  and  inhibitory  reflexes  were  greatly  reduced  or  eliminated  in  the  inter-joint  but  not  the  intra¬ 
joint  reflexes  (Fig.  4).  This  suggests  that  descending  cues  target  interjoint  reflexes  as  the  animal  alters 
coordination  of  joints  to  perform  different  behaviors. 
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Figure  4.  In  intact  insects,  movement  of  the  femoral  chordotonal  organ  evokes  reflex  responses  in  both 
SETi  (motor  neuron  controlling  the  home  FTi  joint)  and  Ds  (motor  neuron  controlling  the  next 
proximal  joint.  However,  lesion  of  neck  connectives  or  circumoesophageal  connectives  (CoCL) 
disrupts  the  excitatory  and  inhibitory  intersegmental  reflexes  to  Ds. 

Processing  in  the  Brain 

At  the  other  end  of  the  system,  research  was  initiated  to  examine  the  descending  commands  that 
generate  these  changes.  Under  the  Eglin  grant,  turning  behaviors  in  insects  with  lesions  to  various 
regions  of  the  brain  were  examined.  These  experiments  clearly  demonstrated  that  the  neuropils  that 
make  up  the  central  body  complex  (CBC)  are  critical  to  control  of  these  transitional  behaviors. 
Lesions  that  involve  the  CBC  showed  significantly  higher  incidence  of  abnormal  turning  behaviors 
such  as  turning  the  wrong  direction  or  failure  to  turn  at  all  after  contacting  an  obstruction.  In  contrast, 


9 


sham  operated  animals  or  animals  that  had  lesions  distant  from  the  CBC  turned  normally.  These  data 
were  reported  in  a  paper  that  was  recently  accepted  to  the  Journal  of  Comparative  Physiology  A 
(Ridgel  et  al„  2006). 


These  data  suggest  descending  commands  that  are  formulated  in  the  CBC  provide  descending 
influences  that  alter  walking  movements.  To  examine  these  commands,  two  new  techniques:  multi¬ 
unit  recording  in  restrained  cockroaches  and  stimulation  in  tethered  insects  were  developed.  Multi¬ 
unit  recording  is  a  new  technique  that  can  now  be  performed  routinely  in  the  brain.  Sixteen-channel 
probes  obtained  from  Michigan  Probes  were  used.  They  were  connected  to  a  Neurolynx  recording 
system,  and  then  inserted  into  various  brain  regions  to  record  brain  activity  from  neurons  at  each 
recording  site.  Cluster  cutting  software  was  then  used  to  identify  individual  units  within  the  raw  data. 
For  example,  in  restrained  animals,  the  antenna  were  moved  back  and  forth  several  times  with  servo 
motors  to  identify  units  that  respond  to  antennal  movement  in  either  direction  or,  more  specifically, 
movement  to  the  right  or  left  side  of  the  animal  (Fig.  5). 


Left  Lateral  Left  Medial  Right  Lateral  Right  Medial 


Figure  5.  Raster  displays  and  peri-event  histograms  of  two  brain  units  recorded  in  the  central  complex 
showing  clear  responses  to  movement  of  either  antenna  either  medially  or  laterally.  Unit  1  also  showed 
inhibition  following  the  positive  response  (red  arrows). 

In  the  stimulation  paradigm,  fine  wire  bundles  are  inserted  into  the  brains  of  tethered  animals.  Regions 
like  the  CBC  or  its  output  regions  (the  lateral  accessory  lobe  -  LAL)  were  stimulated,  and  then  the 
effects  on  movement  were  examined  using  high-speed  video.  Regions  were  found  that  could  evoke 
turning  in  animals  that  are  walking  forward.  In  other  brain  regions,  stimulation  caused  the  insects  to 
simply  walk  faster.  Results  from  both  of  these  paradigms  were  reported  on  at  the  recent  meeting  of  the 
Society  for  Neuroscience  (Pollack  et  al.,  2006).  They  will  serve  as  the  basis  for  more  detailed 
recording  and  stimulation  experiments  that  will  be  conducted  under  an  NSF  grant  to  the  Ritzmann 
laboratory. 


Motor  Activity  Associated  with  Flight  Turns 

Over  the  last  six  months,  work  in  the  Willis  Lab  at  CWRU  has  focused  on  comparing  the  guidance 
algorithms  used  by  walking  and  flying  insects  to  track  odor  plumes.  These  studies  have  included 
behavioral  experiments,  simulation  and  robotic  experiments  with  CWRU’s  Quinn  Lab,  and  the 
ongoing  analysis  of  muscle  activation  patterns  underlying  free-flight  maneuvering  during  odor- 
modulated  navigation. 
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Collaboration  between  the  Willis  Lab  and  the  Quinn  Lab  over  the  course  of  the  research  grant  has 
resulted  in  significant  progress  in  developing  control  algorithms  to  support  chemical  plume  tracking 
behavior  in  autonomous  walking  and  flying  vehicles.  This  research,  using  both  modeling  and  robotic 
approaches,  has  also  led  to  reconsidering  the  existing  hypotheses  for  how  flying  moths  and  walking 
cockroaches  use  olfactory  and  other  sensory  information  to  track  airborne  chemical  plumes.  One  study 
performed  by  a  CWRU  M.S.  student,  developed  an  ultrasonic  anemometer  to  detect  wind  speed  and 
direction,  and  interface  circuitry  to  interface  wind  and  chemical  detection  systems  with  a  wheeled 
koala®  robot  (Bailey  et  al.,  2005).  Ongoing  robotic  and  simulation  modeling  work  has  included  studies 
of  control  algorithms  for  plume  tracking  in  2D  and  3D  space  (Edwards  et  al.,  2005),  the  integration  of 
multiple  sensory  inputs  for  odor  tracking  flight  control  (Rutowski  et  al.,  2006a),  and  an  algorithm  that 
enables  a  flying  agent  to  maintain  its  flight  speed  and  altitude  using  only  optic  flow  input  (Rutkowski 
et  al.,  2006b). 

Behavioral  experiments  performed  in  the  Willis  Lab  have  focused  on  the  response  of  flying  moths  to 
plumes  with  different  spatial  distribution  of  odor  (Morrison  and  Willis,  in  prep.),  and  comparing  these 
responses  to  those  of  walking  cockroaches  to  similar  plumes  (Willis  and  Avondet,  2005).  A  particular 
focus  has  been  on  the  role  of  antennal  postures  and  movements  in  plume  tracking  behavior.  To  date, 
results  have  shown  that  male  moths  tracking  plumes  in  different  wind  speeds,  and  thus  experiencing 
different  amounts  of  aerodynamic  drag  on  their  antennae,  maintain  the  same  antennal  posture 
regardless  of  the  wind  speed.  These  results,  together  with  what  is  known  about  the  sensori-motor 
control  of  airspeed  in  other  insects,  suggests  that  the  moths  are  actively  maintaining  their  antennae  in  a 
specific  posture  possibly  to  detect  the  moths  airspeed.  Associated  work  in  the  laboratory  of  a  Dr.  T. 
Daniel  (Univ.  Washington,  Seattle),  and  funded  by  the  Willis  Lab  ONR-MURI  grant,  suggests  that  the 
antennae  of  Manduca  sexta  may  also  serve  as  Coriolis  force  detectors  feeding  back  to  the  control  of 
yaw.  This  work  is  currently  in  review  for  publication  in  the  journal  Science.  Similarly,  male 
cockroaches  tracking  plumes  of  female  pheromone  maintain  their  antennae  in  a  stereotyped  posture. 
These  results  suggest  that  the  movement  of  antennae  during  olfactory  behaviors  maybe  a  compromise 
between  their  function  as  wind  sensing  organs  and  odor-detecting  organs.  Experiments  in  this  area  are 
ongoing.  Recent  experiments  on  multi-sensory  inputs  supporting  chemical  plume  tracking  in  walking 
male  cockroaches  has  shown  that  individuals  with  no  visual  inputs  can  track  plumes  just  as  well  as 
intact  individuals.  In  these  experiments,  the  performance  of  intact  males  were  compared  to  those  that 
had  their  large  compound  eyes,  and  simple  occelli  covered  with  black  paint.  Males  with  only  their 
compound  eyes  painted  performed  as  well  as  intact  males.  However,  males  that  had  their  simple  eyes 
painted  walked  more  slowly,  and  the  only  individuals  in  the  entire  experimental  design  that  were 
unable  to  track  the  plume  to  its  source  were  those  with  their  simple  eyes  painted. 

Recent  results  from  ongoing  analysis  of  recordings  of  muscle  activity  patterns  from  freely  flying  moths 
during  odor  tracking  flight  have  revealed  an  association  between  bi-laterally  symmetrical  muscle 
activation  and  lateral  movement  during  turning  maneuvers.  It  has  been  observed  that  the  indirect 
elevators  of  the  wings,  that  cause  the  upstroke  of  each  wingbeat  cycle,  shift  the  timing  of  their 
activation  each  wing  stroke  in  an  asymmetric  way  during  the  zigzag  turning  that  characterizes 
pheromone  tracking  in  these  moths.  The  central  nervous  system  (CNS)  advances  and  delays  the 
activation  of  these  muscles  in  a  predictable  way  associated  with  rightward  and  leftward  turns.  The 
commonly  accepted  model  of  flight  maneuvering  in  insects  like  the  moth  M.  sexta,  is  that  the  indirect 
flight  muscles  are  not  involved  in  steering.  CWRU  results  suggest  that  this  is  not  the  case.  Ongoing 
analysis  will  increase  our  sample  size  and  enable  the  comparisons  across  multiple  behavioral  and 
motor  performances  of  the  same  individual. 
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Implementation  of  Biologically  Inspired  Control  into  Robotic  Systems 

Neural  control  of  a  robotic  leg:  One  particularly  interesting  project  that  has  begun  in  the  past  year  is 
the  implementation  of  a  neurally-based  control  system  for  insect  leg  movement  in  a  robot  model.  This 
model  will  serve  as  a  means  to  test  hypotheses  regarding  the  neural  control  of  leg  movement,  and  it 
additionally  holds  promise  for  the  reduction  of  control  complexity  in  legged  robots. 

The  control  system  was  developed  by  Orjan  Ekeberg  at  the  Royal  Institute  of  Sweden,  with  Marcus 
Blumel  and  Ansgar  Buschges  in  Cologne,  Germany  based  upon  neurobiological  experiments  on  stick 
insects.  It  is  the  most  complete  description  of  the  local  reflexes  that  have  been  documented  for  the 
stick  insect,  and  has  successfully  moved  the  legs  of  a  dynamic  model  insect.  The  control  system 
consists  of  a  bistable  pattern  generator  for  each  joint,  which  changes  states  (e.g.  from  flexion  to 
extension)  based  on  sensory  information  from  the  leg.  This  sensory  information  is  of  course  in  turn 
influenced  by  the  activation  of  the  joints  and  the  interaction  of  the  leg  with  the  environment.  This 
control  method,  which  we  term  Sensory  Coupled  Action  Switching  Modules  (SCASM),  is  shown  in 
Fig.  6  and  described  in  (Lewinger  et  al.  2006,  Ekeberg  et  al.  2004). 


Figure  6.  Graphical  representation  of  the  bi-stable  pattern  generators  controlling  each  joint,  showing 
the  flow  of  sensory  information  between  joints.  The  transition  rules  represent  the  controller  used  for 
three  separate  walking  systems;  only  the  thresholds  differ  between  the  models. 

A  collaboration  was  initiated  with  the  Cologne  group  that  proposes  developing  robotic  stick  insect  and 
cockroach  legs  to  be  controlled  by  this  model.  Upon  completion,  the  dedicated  robotic  models  to  test 
hypotheses  such  as  our  notion  of  how  inside  turning  occurs  can  then  be  used.  At  this  time  the  stick 
insect  leg  has  been  constructed  (Fig.  7,  left)  and  can  make  stepping  movements  under  control  of  the 
stick  insect  model. 
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Two  CWRU  students  have  taken  this  leg  and  a  two-leg  robotic  test  platform  (Fig  7,  right)  to  Cologne 
to  work  with  the  Biischges  laboratory  in  improving  the  leg  controller,  and  collaborating  on  the 
development  of  a  unified  experimental  interface  for  the  various  simulations  and  robotic  models  under 
development.  It  is  believed  that  these  dedicated  robotic  models  will  provide  very  powerful  tools  for 
testing  neurobiological  hypotheses. 

For  this  project,  previous  lab  experience  with  RTLinux  has  been  leveraged  to  build  software  capable  of 
implementing  this  type  of  control  scheme,  which  represents  biological  motor  systems  for  robot  control 
with  real-time  parameter  modification.  Feed-forward  torque  control  using  small  electrical  servomotors 
for  simulation  of  muscle  function  have  also  been  implemented. 


Fig.  7  LEFT:  Single-leg  scale  model  of  a  stick  insect  middle  leg  for  biological  experimentation.  This 
view  is  from  the  front;  the  bar  extending  to  the  bottom  right  indicates  the  orientation  of  the  thorax  in 
the  horizontal  plane.  RIGHT:  Two-leg  platform  for  implementation  of  SCASM  in  low-power  mobile 
robots. 

The  same  SCASM  architecture  has  successfully  produced  walking  motions  which  adapt  to 
environmental  changes  in  three  systems:  the  original  simulation,  the  single-leg  robotic  stick  insect 
model  and  a  two-leg  robot  test  platform.  In  the  animal  this  control  architecture  provides  muscle 
commands;  all  three  of  the  artificial  systems  have  required  simple  muscle  models  to  function 
successfully. 

It  has  been  shown  that  very  simple  muscle  models  will  allow  system  operation  (paper  submitted  to 
ICRA  2007)  (See  Fig.  8).  Further  research  will  investigate  the  extent  to  which  increasing  muscle 
model  complexity  would  be  useful  from  a  robot  control  point  of  view. 
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Min-min  duration,  Piecewise- Constant,  median  1.44221  iqr  0.335271 


Min-min  delta,  Piecewise-Constant  median  0  rqr  2.61609 


Fig.  8  A  demonstration  of  the  improvement  in  forward  stepping  when  a  simple  linear  muscle  model  is 
implemented  at  one  joint  (bottom  graphs).  LEFT:  Histograms  showing  the  distributions  of  durations  of 
the  entire  step  cycle,  as  measured  from  one  minimum  of  ThC  angle  to  the  next  (the  point  at  which  the 
foot  is  lifted  and  swing  begins).  The  medians  do  not  differ  significantly  according  to  the  Wilcoxon  test 
(p=0.22),  and  the  distributions  differ  with  p  =  lxl O'8  according  to  the  Kolmogorov- Smirinov  test.  Note 
the  bimodal  behavior  in  the  system  without  the  linear  muscle  model.  RIGHT:  The  distributions  of 
difference  of  the  minimum  ThC  angle  from  step  to  step.  The  medians  do  not  differ  significantly 
according  to  the  Wilcoxon  test  (p=0.14),  but  the  distributions  differ  with  p  =  3.7xl0'5  according  to  the 
Kolmogorov-Smirinov  test.  The  lack  of  the  wide  tails  on  data  taken  using  the  linear  muscle  model 
indicates  a  more  consistent  placement  of  the  foot  at  transition  from  stance  to  swing. 


Cockroach  EMGs  control  robot  leg:  Several  other  robot  control  system  projects  at  CWRU 
implement  the  use  of  cockroach  EMGs  to  control  a  robot  leg.  A  simple  open-loop  transformation  from 
EMG  to  inlet  valve  commands  for  Braided  Pneumatic  Actuators  (BPAs)  has  been  developed  and  tested 
on  walking  behavior,  driving  artificial  muscles  at  the  CTr  joint  of  a  robot  model  (Fig.  9,  left)  with 
EMGs  from  corresponding  muscles  in  the  animal.  It  uses  hardware-in-the-loop  error  minimization  on 
an  RT-Linux  system  to  identify  appropriate  transformation  parameters  which  make  the  EMG-driven 
robot  motion  most  closely  resemble  the  observed  animal  behavior  (Rutter,  Mu  et  al.  2006,  and  a  paper 
submitted  to  ICRA  2007)  (Fig.  10).  This  work  suggests  that  this  approach  holds  promise  for  use  as  a 
tool  in  the  investigation  of  (insect)  musculoskeletal  systems.  CWRU  is  currently  conducting  an  initial 
investigation  into  what  changes  are  necessary  in  this  transformation  to  represent  turning  behavior,  with 
the  expectation  that  this  could  improve  the  understanding  of  changes  between  these  behaviors  in 
properties  of  the  animal’s  neuromuscular  transform.  The  simplicity  of  the  transformation  also  suggests 
that  in  cases  where  EMGs  are  used  as  a  system  control  input,  the  use  of  BPAs  can  reduce  control 
system  complexity. 

For  this  project,  research  was  completed  on  functional  cockroach  joint  anatomy  both  for  determining 
the  joint  geometry  and  range  of  motion  for  this  model,  and  the  degrees  of  freedom  necessary  for  a 
robotic  cockroach  leg  model  based  on  the  stick  insect  model  above.  Together  with  results  from  the 
EMG  modeling  work,  these  data  have  suggested  further  improvements,  several  of  which  will  be 
implemented  in  what  will  be  the  most  functionally  accurate  robotic  model  of  a  cockroach  leg  yet  built 
(Fig.  9,  right). 
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Fig.  9:  LEFT:  a  close-up  of  the  right  middle  leg  of  Robot  V  as  modified  for  this  work.  CENTER:  a 
LEGO  kinematic  model  which  uses  the  results  of  the  functional  anatomy  studies.  One  copy  of  this 
model  resides  in  the  Ritzmann  laboratory,  the  other  in  the  Quinn  laboratory.  RIGHT :  This  functionally 
accurate  robot  model  of  a  Blaberus  middle  right  leg  will  make  use  of  cabled  artificial  muscles  for 
actuation,  in  order  to  allow  joint  ranges  of  motion  observed  in  the  animal. 


Q uarter-1  first  point  joint  angles  Quarter-2  best  point  joint  angles 


Fig.  10:  EMG-driven  joint  angles  before  (left)  and  after  (right)  the  error  minimization  used  to  find 
appropriate  transformation  parameters. 


SUMMARY 

CWRU  has  made  significant  progress  in  describing  transitional  behaviors  in  insect  walking  and  in 
flight.  Quantitative  descriptions  of  the  complex  decisions  that  a  cockroach  makes  in  deciding  to  climb 
over  or  tunnel  under  a  barrier  has  positioned  researchers  for  examining  discrete  brain  lesions  in  order 
to  pin  point  where  and  how  these  decisions  are  made.  Documentation  of  turning  movements 
developed  hypotheses  regarding  how  descending  cues  might  alter  local  reflexes  to  turn  the  animal 
while  retaining  stability.  Lesions  in  the  brain  and  subsequent  recordings  are  beginning  to  demonstrate 
how  the  descending  commands  might  be  formulated  within  the  brain.  Finally,  a  robotic  leg  that  was 
developed  shows  great  promise  as  a  hardware  model  of  the  control  system,  and  could  play  a  critical 
role  in  testing  our  hypotheses  at  a  systems  level. 
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The  results  that  were  obtained  have  positioned  CWRIJ  to  move  ahead  with  new  funding  initiatives. 
The  brain  recording  and  stimulation  projects  that  were  initiated  under  this  recently  completed  effort, 
are  now  funded  by  an  NSF  grant  to  the  CWRU  Ritzmann  Laboratory.  Meanwhile,  the  projects  that 
were  undertaken  to  understand  the  alterations  that  occur  at  the  local  control  level  in  response  to 
descending  commands  is  being  pursued  in  a  follow-on  proposal  to  AFOSR.  This  proposal  also 
includes  development  of  the  robotic  leg  hardware  model.  These  two  projects,  which  spun-off  of  the 
Eglin  funded  research,  will  continue  to  complement  each  other  as  CWRU  continues  to  strive  for  an 
understanding  of  how  insects  control  movement  through  complex  terrain. 
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